A storage system typically comprises one or more storage devices into which information may be entered, and from which information may be obtained, as desired. The storage system includes a storage operating system that functionally organizes the system by, inter alia, invoking storage operations in support of a storage service implemented by the system. The storage system may be implemented in accordance with a variety of storage architectures including, but not limited to, a network-attached storage environment, a storage area network and a disk assembly directly attached to a client or host computer. The storage devices are typically disk drives organized as a disk array, wherein the term “disk” commonly describes a self-contained rotating magnetic media storage device. The term disk in this context is synonymous with hard disk drive (HDD) or direct access storage device (DASD).
The storage operating system of the storage system may implement a high-level module, such as a file system, to logically organize the information stored on volumes as a hierarchical structure of data containers, such as files and logical units (LUs). For example, each “on-disk” file may be implemented as set of data structures, i.e., disk blocks, configured to store information, such as the actual data for the file. These data blocks are organized within a volume block number (vbn) space that is maintained by the file system. The file system may also assign each data block in the file a corresponding “file offset” or file block number (fbn). The file system typically assigns sequences of fbns on a per-file basis, whereas vbns are assigned over a larger volume address space. The file system organizes the data blocks within the vbn space as a “logical volume”; each logical volume may be, although is not necessarily, associated with its own file system.
A known type of file system is a write-anywhere file system that does not overwrite data on disks. If a data block is retrieved (read) from disk into a memory of the storage system and “dirtied” (i.e., updated or modified) with new data, the data block is thereafter stored (written) to a new location on disk to optimize write performance. A write-anywhere file system may initially assume an optimal layout such that the data is substantially contiguously arranged on disks. The optimal disk layout results in efficient access operations, particularly for sequential read operations, directed to the disks. An example of a write-anywhere file system that is configured to operate on a storage system is the Write Anywhere File Layout (WAFL®) file system available from NetApp, Inc. Sunnyvale, Calif.
The storage system may be further configured to operate according to a client/server model of information delivery to thereby allow many clients to access data containers stored on the system. In this model, the client may comprise an application, such as a database application, executing on a computer that “connects” to the storage system over a computer network, such as a point-to-point link, shared local area network (LAN), wide area network (WAN), or virtual private network (VPN) implemented over a public network such as the Internet. Each client may request the services of the storage system by issuing access requests (read/write requests) as file-based and block-based protocol messages (in the form of packets) to the system over the network.
A plurality of storage systems may be interconnected to provide a storage system architecture configured to service many clients. In some embodiments, the storage system architecture provides one or more aggregates and one or more volumes distributed across a plurality of nodes interconnected as a cluster. The aggregates may be configured to contain one or more volumes. The volumes may be configured to store content of data containers, such as files and logical units, served by the cluster in response to multi-protocol data access requests issued by clients.
Each node of the cluster may include (i) a storage server (referred to as a “Data blade”) adapted to service a particular aggregate or volume and (ii) a multi-protocol engine (referred to as an “N-blade” Network blade) adapted to redirect the data access requests to any storage server of the cluster. In the illustrative embodiment, the storage server of each node is embodied as a disk element (Data blade) and the multi-protocol engine is embodied as a network element (Network blade). The Network blade receives a multi-protocol data access request from a client, converts that access request into a cluster fabric (CF) message and redirects the message to an appropriate Data blade of the cluster.
The nodes of the cluster may be configured to communicate with one another to act collectively to increase performance or to offset any single node failure within the cluster. Each node in the cluster may have a predetermined failover “partner” node that may take over/resume storage functions of the node upon failure of the node. When a node failure occurs (where the failed node is no longer capable of processing access requests for clients), the access requests sent to the failed node may be re-directed to the partner node for processing. As such, the cluster may be configured such that a partner node may take over the work load of a failed node. As used herein, a local/source node may have data and metadata that is mirrored/copied to a remote/destination node in the cluster storage system (as discussed below). The remote node may comprise a predetermined failover partner node of the local node. As used herein, various components residing on the local node may likewise be referred to as a local component (e.g., local memory, local de-staging layer, etc.) and various components residing on a remote node may likewise be referred to as a remote component (e.g., remote memory, remote de-staging layer, etc.).
A cluster provides data-access service to clients by providing access to shared storage (comprising a set of storage devices). Typically, clients will connect with a node of the cluster for data-access sessions with the node. During a data-access session with a node, a client may submit access requests (read/write requests) that are received and performed by the node. For the received write requests, the node may produce write logs that represent the write requests and locally store the write logs to a volatile storage device (from which, the node may at a later time perform the write logs on the storage devices).
To ensure data consistency and provide high data availability, the write logs may also be stored to two non-volatile storage devices. Typically, the write logs of the node may be locally stored to a non-volatile storage device and also be stored remotely to a non-volatile storage device at a partner node (sometimes referred to herein as mirroring data to a remote node). As such, if the local node fails, the remote partner node will have a copy of the write logs and will still be able to perform the write logs on the storage devices. Also, if the write logs stored at the partner node is corrupted or lost, the write logs stored locally in the non-volatile storage device at the local node can be extracted/retrieved and used to perform the write logs on the storage devices.
As such, data in a local non-volatile storage device at a local node may be mirrored to a remote non-volatile storage device of a remote node to provide failover protection (e.g., in case the local node crashes) and high availability of data in the cluster storage system. The mirrored data may comprise write logs, or any other data that is to be stored to the non-volatile storage devices.
Typically, for a group of related data sets (e.g., data sets X, Y), there may also be a metadata set (e.g., metadata set Z) that describes each of the related data sets (e.g., metadata set Z describes data sets X, Y), the metadata set to also be stored to the local and remote non-volatile storage devices. As used herein, a “related group” of data and metadata sets may comprise one or more data sets and one metadata set that describes and is associated with each of the one or more data sets. For example, the data sets of a related group may comprise data sets X, Y and metadata set Z, where metadata set Z specifies that there are 2 valid data sets. A “valid” data set may comprise user/client data that is pending to be stored to the local and remote non-volatile storage devices. The ratio of data sets to a metadata set produced may vary depending on the mirroring client. For example, a particular mirroring client may continually produce and send one metadata set for every two data sets, whereby each related group comprises two data sets and one metadata set.
Metadata sets may be produced at the local node to assist in the processing of the data sets, the metadata sets also being mirrored to the remote node. However, additional software and hardware resources are expended to process the metadata sets and the metadata sets cause additional network congestion between the local and remote nodes, consuming the network connection's valuable data bandwidth. Since metadata sets do not comprise user/client data, the metadata sets may be considered overhead that should be limited as much as possible.